None Notebook

This notebook contains material from cbe61622; content is available on Github.

< 10.0 Projects | Contents | A.0 Python Source Library >

Open in Colab

Download

10.10 Computer Vision Case Study: Finding Particles in Images

The goal of this notebook is to develop a python class for the purpose of counting and labeling flourescent particles captured in images from a prototype for a low-cost medical diagnostic device.

In this notebook we'll consider traditional image processing techniques .. i.e., those that directly operate on an image to extract scientific information without resort to machine learning techniques. We'll save that imoportant discussion for another time.

10.10.1 Books and References

Computer Vision Textbooks

Nixon, Mark, and Alberto Aguado. Feature extraction and image processing for computer vision. Academic press, 2019. link

Szeliski, Richard. Computer vision: algorithms and applications. Springer Science & Business Media, 2010. [Hesburgh Library][Szeliski Web Page and Materials][Preprint of 2nd Edition]

Programming Books

Howse, Joseph, and Joe Minichino. Learning OpenCV 4 Computer Vision with Python 3: Get to grips with tools, techniques, and algorithms for computer vision and machine learning. Packt Publishing Ltd, 2020.

Villán, Alberto Fernández. Mastering OpenCV 4 with Python: A practical guide covering topics from image processing, augmented reality to deep learning with OpenCV 4 and Python 3.7. Packt Publishing Ltd, 2019.

Pajankar, Ashwin. Raspberry Pi Computer Vision Programming: Design and implement computer vision applications with Raspberry Pi, OpenCV, and Python 3. Packt Publishing Ltd, 2020. Amazon

Papers

Coelho, L.P. 2013. Mahotas: Open source software for scriptable computer vision. Journal of Open Research Software 1(1):e3, DOI: http://dx.doi.org/10.5334/jors.ac

Van Noorden, Richard. "Publishers launch joint effort to tackle altered images in research papers." Nature (2020). https://doi.org/10.1038/d41586-020-01410-9

Blogs and Postings

10.10.2 Python Packages for Computer Vision

See https://www.analyticsvidhya.com/blog/2021/04/top-python-libraries-for-image-processing-in-2021/ for a survey of Python libraries for image processing.

Standard Python Libaries

One of the very nice aspects of image processing with Python is the wide adoption of basic NumPy arrays to represent image data. This facilitates the use of methods from multiple packages for a particular project.

Pure Python Libraries

Full Featured Computer Vision Packages

Photo Conversion and Management Tools

Outdated or deprecated libraries

These are included here so you know what not to use for your projects.

10.10.2.1 File Formats

For this case study we will be developing techniques to analyze images stored as computer files. Later we may consider applying these techniques to a live video stream, but for now we'll use stored images.

Things to know about image files.

General recommendations for scientific use.

10.10.2.2 Image Metadata

Image files also carry meta-data providing additional information about the image. There several different standards for meta data, the most common being exif data embedded in the image file, or a companion .xmp file hold data in the Extensible Metadata Platform format. exiftool is a command line tool included with many operating systems.

What's not present in this example? (Hint: What colors are being displayed?)

Here's the exif data for a photograph prepared through a typical photographer's workflow: RAW --> Noise Reduction --> Adobe Lightroom --> .jpg for export. Carefully look for the color space data.

10.10.2.3 Calibrating Monitors

Accurate color reproduction requires color management from source to display. At each Color management is now a standard part of most operating systems

10.10.3 Developing a Process

Our approach ...

10.10.3.1 Python Imports

We track overall code dependencies by consolidating imports into this cell. Note that we'll be using elements from multiple packages by relying on the underlying NumPy representation of images to hold the current state of the process.

10.10.3.2 Reading images

As a first step, read the image, convert to rgb scale, and display. All of these packages have a means of reading raster file images in common formats. There are small (and sometimes frustrating) differences among them. Here we use the Matplotlib imread() method which reads and returns a numpy array.

The array will typical have 2 or 3 dimensions $(h, w, d)$ where $h$ and $w$ are image height and width, and $d$ is pixel depth.

Observations:

10.10.3.3 Cropping

10.10.3.4 Channels and Histograms

An image is comprised of one or more channels

Histograms are a tool for analyzing the distribution of gray levels in a channel. It's a powerful tool for controlling exposure and processing images for presentation.

10.10.3.5 Creating a composite channel

We see the blue leds used to excite the flourophores bleed over into the green channel. It would be best if this could be corrected in the experiment, perhaps by positioning a bandpass filter in front of the leds. What we will attempt here is subtract a multiple of blue channel from the green channel, followed by exposure adjustments. The goal is to provide a cleaner image for doing particle labeling and counting.

By trial and error, we find a weighted difference of the green and blue channels, and a rescaling of the tone curve that retains the particles and reduces background interference.

10.10.3.6 Histogram equalization

Observations

10.10.3.7 Blur filter

10.10.3.8 Thresholding/Segmentation

The purpose of threshold is to isolate the features of interest from background noise.

https://docs.opencv.org/3.4/d7/d4d/tutorial_py_thresholding.html

10.10.3.9 Morphological Transformation

The next goal is to remove noise and to separate particles.

10.10.3.10 Finding Objects

http://pageperso.lif.univ-mrs.fr/~francois.denis/IAAM1/scipy-html-1.0.0/tutorial/ndimage.html

10.10.3.11 Finding and Displaying Particles

10.10.3.12 What did we learn about our application?

10.10.4 Particle Labeling Classes

To facilitate embedded use in a device, the next step is to consolidate these procedures into a class.

10.10.5 Demonstrations

10.10.5.1 Thresholding

< 10.0 Projects | Contents | A.0 Python Source Library >

Open in Colab

Download